Stationary Probabilities of Markov Chains
نویسنده
چکیده
In this paper, based on probabilistic arguments, we obtain an explicit solution of the stationary distribution for a discrete time Markov chain with an upper Hessenberg time stationary transition probability matrix. Our solution then leads to a numerically stable and eecient algorithm for computing stationary probabilities. Two other expressions for the stationary distribution are also derived, which lead to two alternative algorithms. Numerical analysis of the algorithms is given, which shows the reliability and eeciency of the algorithms. Examples of applications are provided, including results of a discrete time state dependent batch arrive queueing model. The idea used in this paper can be generalized to deal with Markov chains with a more general structure.
منابع مشابه
Computing 2-Step Predictions for Interval-Valued Finite Stationary Markov Chains
Markov chains are a useful tool for solving practical problems. In many real-life situations, we do not know the exact values of initial and transition probabilities; instead, we only know the intervals of possible values of these probabilities. Such interval-valued Markov chains were considered and analyzed by I. O. Kozine and L. V. Utkin in their Reliable Computing paper. In their paper, they...
متن کاملUniform Stability of Markov Chains*
By deriving a new set of tight perturbation bounds, it is shown that all stationary probabilities of a finite irreducible Markov chain react essentially in the same way to perturbations in the transition probabilities. In particular, if at least one stationary probability is insensitive in a relative sense, then all stationary probabilities must be insensitive in an absolute sense. New measures...
متن کاملA Successive Lumping Procedure for a Class of Markov Chains
A class of Markov chains we call successively lumpable is specified for which it is shown that the stationary probabilities can be obtained by successively computing the stationary probabilities of a propitiously constructed sequence of Markov chains. Each of the latter chains has a(typically much) smaller state space and this yields significant computational improvements. We discuss how the re...
متن کاملStationary Probabilities of Markov Chains with Upper Hessenberg Transition Matrices
In this paper, based on probabilistic arguments, we obtain an explicit solution of the stationary distribution for a discrete time Markov chain with an upper Hessenberg time stationary transition probability matrix. Our solution then leads to a numerically stable and efficient algorithm for computing stationary probabilities. Two other expressions for the stationary distribution are also derive...
متن کاملOn the Continuous Diffusion Approximation of Some Discrete Markov Chains
The diffusion approximation of some Markov chains with Binomial-like transition probabilities is considered and their qualitative behavior is analyzed. The stationary distribution, attainability of boundary and estimation of parameters are investigated. Some examples are reported.
متن کامل